Reconnecting Host to Contrail Windows CI cluster

This document describes a procedure required to reconnected a VMware ESXi host to the Contrail Windows CI cluster. Reconnecting a host to CI cluster is understood as a set to steps leading to usage of this host in Contrail Windows CI.

Described procedure must be performed only when:

• a host with reinstalled ESXi must be reconnected to the cluster,
• a host with replaced hard drive must be reconnected to the cluster.

Prerequisites

Requirements for CI admin:

• Access to Contrail Windows CI infrastructure.
• Credentials to Contrail Windows CI VMware cluster.
• Browser with Flash support, preferably Chrome.

Requirements for a reconnected host:

• Host must be connected to vCenter.

Steps

1. Open the vSphere Web Client and login using credentials provided by Contrail Windows CI team.
2. In Navigator pane select Hosts and clusters tab.
3. Expand CI-DC datacenter entry.
4. Expand WinCI cluster entry.
5. Find a host entry representing a being reconnected.
   • If a host cannot be found, it is not connected to vCenter. In that case, please contact infrastructure team.
   • if a host is connected to vCenter, but is not a part of WinCI cluster:
     • Right click on the host and select Move To... option.
     • In the Move To... window, expand CI-DC datacenter.
     • Click on WinCI cluster.
     • Click OK button.
     • If a dialog Move Host into This Cluster comes up, select Put all of this host's virtual machines in the cluster's root resource pool option.
     • Click Ok.
6. If a host is marked as being in maintenance mode:
   • Right click on a host entry and select Maintenance mode > Exit maintenance mode.
7. Click on the host entry.
8. Host networking reconfiguration:
   • In the middle pane select Configure tab.
   • Select Virtual switches from the list on the left.
   • Select vSwitch0 from the virtual switches list.
   • Click on VM Network port group in the bottom.
   • Click Edit settings button.
   • Type in VM-Network in the Network label input and click Ok button.
   • Select VMkernel adapters from the list on the left.
   • Select on vmk0 adapter from the adapter list.
   • Click Edit settings button.
   • In Port properties wizard page, check vMotion checkbox. Click Ok button.
9. In Navigator pane select Storage tab.
   • Right click on NFS-Datastore and select Mount Datastore to Additional Hosts.
   • Mark a checkbox next to a host entry representing a being reconnected.
   • Click OK.
   • Right click on winci_nfsbackup and select Mount Datastore to Additional Hosts.
   • Mark a checkbox next to a host entry representing a being reconnected.
   • Click OK.
10. In Navigator pane select Hosts and clusters tab.
11. Find a host entry representing a being reconnected and click on it.
12. Reconfiguring ESXi logs location to a remote location.
    • In the middle pane select Configure tab.
    • Select Advanced System Settings from the list on the left.
    • Click Edit button in the upper right corner.
    • Type in logDir in the Filter input box and press Enter.
    • Change the value of Syslog.global.logDir to [NFS-Datastore] logs.
    • Mark the Enabled checkbox in Syslog.global.logDirUnique.
    • Click Ok button.
    • Type in logDir in the Filter input box and press Enter.
    • Look through the filtered list and verify that provided options are saved.
    • To verify that logs are stored in remote location, perform the following steps:
      • In the Navigator pane select Storage tab.
      • Click on NFS-Datastore.
      • In the main pane select Files tab.
      • Navigate to logs directory.
      • Navigate to a directory named with host's hostname.
      • ESXi logs should be stored in this directory.
      • Go back to host's entry in Hosts and clusters tab in Navigator pane.
13. Cleanup of orphaned VMs must be performed.
    • In the middle pane select VMs tab.
    • In the VM table, click Name header to sort VMs by name in ascending order.
    • For each VM perform a following process:
      • Determine if the VM is critical.
        • Refer to List of all important VMs in Windows CI and check if this VM is on the list marked as CRITICAL.
          • In case of any doubts, please contact Contrail Windows CI team.
      • If the VM is marked as Orphaned and it is not critical to CI functioning:
        • Left click on the VM.
        • In the middle pane, scroll down to Related Objects window and check if VM is still located on local datastore.
        • If it is the VM must be reregistered and then removed.
          • Please follow steps in Reregistering a VM to reregister the VM.
          • Right click on the VM and select Delete from disk option.
        • If it is not, the VM can be safely removed from inventory.
          • Right click on the VM.
          • Select All Virtual Infrastructure Actions > Remove from Inventory option.
      • If a VM is marked as Orphaned and it is critical to CI functioning:
        • Left click on the VM.
        • In the middle pane, scroll down to Related Objects window and check if VM is still located on local datastore.
        • If it is the VM must be reregistered.
          • Please follow steps in Reregistering a VM reregister the VM.
          • Power on the VM.
        • If it is not, the VM must be restored from backups.
          • Please refer to Infrastructure backups.
14. Assign host's datastores to datastore clusters.
    • In the middle pane select Datastores tab.
    • Please contact a Contrail Windows CI team regarding datastore - cluster association and perform the following steps:
      • Right click on a datastore.
      • Select Move to option.
      • In the Move To... window select a datastore cluster based on information from Contrail Windows CI team.

After reconnecting

After host is reconnected to Contrail Windows CI cluster, please consider the following:

• Migrate some of ci-builder-* nodes to this new host.
  • TODO: document

Appendices

Reregistering a VM

This guide assumes that the user is located in the VM window.

1. Take note of a datastore names presented in Related Objects windows.
2. Right click on the VM in the list on the left.
3. Select All Virtual Infrastructure Actions > Remove from Inventory option.
4. In the Navigation pane, left click Storage tab.
5. In the datastore list, click on the one of datastores listed previously in Related Objects window.
6. Navigate to Files tab.
7. Locate a folder named like a VM and enter it.
   • If it cannot be found on this datastore, try another one.
8. Right click on [VM-NAME].vmx file and select Register VM option.
   • The Register Virtual Machine wizard should show up.
   • In the Name and Location step select a suitable folder to put a VM in.
   • Click Next.
   • In the Host / Cluster step select a WinCI cluster.
   • Click Next.
   • In the Ready to Complete step click Finish button.